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Abstract 

We consider some discrete and continuous dynamics in a Banach space involv- 
ing a non expansive operator J and a corresponding family of strictly contract- 
ing operators $(A, x) := XJ(^^-x) for A G]0, 1]. Our motivation comes from the 
study of two-player zero-sum repeated games, where the value of the n-stage game 
(resp. the value of the A-discounted game) satisfies the relation v n = &(-^,v n -i) 
(resp. v\ = $(A, v\)) where J is the Shapley operator of the game. We study the 
evolution equation u'(t) = J(u(t)) — u(t) as well as associated Eulerian schemes, 
establishing a new exponential formula and a Kobayashi-like inequality for such tra- 
jectories. We prove that the solution of the non-autonomous evolution equation 
u'(t) = &(\(t),u(t)) — u(t) has the same asymptotic behavior (even when it di- 
verges) as the sequence v n (resp. as the family v\) when \(t) = l/t (resp. when 
A(i) converges slowly enough to 0). 

1 Introduction 



The topic of the asymptotic behavior of trajectories defined through nonexpansive map- 
pings in Banach spaces arise in numerous domains such as nonlinear semigroups the- 
ory [3, 6, 7, 12, 13, 17, 21], game theory [15, 19, 20, 24-26] as well as in discrete events 
systems [9-11]. 

Given a nonexpansive function J from a Banach space X to itself, evolution equation 

U\t) = J(U(t))-U(t) (1.1) 
is a particular case of the widely-studied 

U'(t) e -A(U(t)) 

for a maximal monotone operator A. Typically, the study of the asymptotics for such 
evolution equation and its Eulerian and proximal discretizations has been made in Hilbert 
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spaces [6] or at least assuming some geometric properties in the case of Banach spaces 
[15,21]. Another usual assumption is the non emptiness of the set A -1 (0). 

On the other hand, in the framework of two-person zero-sum games repeated in discrete 
time, the values v n and v\ of the n-stage (resp. A-discounted) game satisfy respectively: 

u„ = —^ = $ ~,u„_i 1.2 
n \n J 

v x = $(\,v x ) (1.3) 

where J is the so-called Shapley operator of the game and $(A,x) := AJ(^x). This 
operator J is nonexpansive for the uniform norm, hence A = I — J is a maximal monotone 
operator in the sense of [12]. However two unusual facts appears in the study of the 
asymptotics of those values: first A _1 (0), the set of fixed points of J, is generally empty. 
Another difficulty lies in the lack of smoothness of the unit ball -By-y^, which might induce 
oscillations of the discrete trajectories defined above [15]. 

The purpose of this paper is to investigate the relation between several discrete and 
continuous dynamics in Banach spaces. Because our motivation comes from this game- 
theoretic framework, we neither make any geometrical assumptions on the unit ball, nor 
suppose non emptiness of A _1 (0). In continuous time, dynamics that we will consider are 
(1.1) as well as non autonomous evolution equations of the form 

u\t) = $(A(t),u(t)) -u(t) (1.4) 

for some parametrizations A. We establish that the quantities defined in (1.2) and (1.3) 
behave asymptotically as the solutions of these various evolution equations. Surprisingly 
this is true not only when there is convergence; even when they oscillate we prove that 
discrete and continuous trajectories remain asymptotically close. 

Section 2 is devoted to definitions and basic results. In Section 3 we study the re- 
lation between the solution U of evolution equation (1.1) and related Eulerian schemes, 
establishing in particular that \\v n — ^^|| converges to 0. In the process we prove that 
some classical results (e.g. exponential formula [7], Kobayashi inequality [13]) involving 
the proximal trajectories for a maximal monotone operator A have an Eulerian explicit 
counterpart in the case A = I — J. In Section 4 we consider the non autonomous equation 
(1.4). We show that for A(t) = | the solution behave asymptotically as the sequence v n , 
and that when A converges slowly enough to the solution behave asymptotically as the 
family v\. 

2 Discrete time model 
2.1 Nonexpansive operators 

Let (X, || • ||) be a Banach space, and J a nonexpansive mapping from X into itself : 

\\J(x)-J(y)\\<\\x-y\\ W(x,y)eX 2 . 
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We define, for n G N and A G]0, 1], 

K = J(K-i) = «/"(0) (2.1) 
V x = J((1-X)V X ) (2.2) 

Notice that V\ is well-defined because J((l — A)-) is strictly contracting, hence has a 
unique fixed point. 

Example 2.1 For any c6l, £/ie mapping J from R to ifeel/ defined by J(x) = x + c is 
nonexpansive. In that case, V n = nc and V\ — 

These quantities being unbounded in general (see above), we also introduce their normal- 
ized versions 

v n = — (2.3) 

n 

vx = XV X (2.4) 

In the previous example, one gets v n = v x = c for all n and A. In general it is easy to 
prove that these normalized quantities are bounded: 

Lemma 2.2 For any n E N and A g]0, 1], 

Kll < 11^(0)11 (2.5) 
INI < \\J(0)\\- (2-6) 

Proof. Since J is non expansive, 

IIK-K-ill = \\J(V n -i)- J(V n - 2 )\\ < ||K-i-K- 2 ||. 

By induction this implies that 

||K|| <n||yi||=n||J(0)||. 
On the other hand, again using the fact that J is non expansive, 

\\Vx\\-\\J(0)\\ < II^a-J(0)|| 

= ||J((1-A)V A )-J(0)|| 

< (1-A)||Va|| 

and so 

INI = A||v A || < ||J(o)||. 

■ 

To underline the link between the families {f n } nG N and {v a}ag]o,i] ^ is also of interest 
to introduce the family of strictly contracting operators $(A, ■), A g]0, 1], defined by 

$(A,x) = A J (~)~~ x j ■ ( 2 - 7 ) 



The function $(A, •) can be seen as a perturbed recession function of J: because of the 
nonexpansiveness of J, 

r x\ J (tx) 



lim $(A, x) — lim A J (^) — lim — ^ (2.i 

A^O A^O \A/ t^+oo t 



which is the definition of the recession function of J [23]. 
The quantities v n and v\ then satisfy the relations 



$ ( -,v n -i ) ; v Q = (2.9) 



v x = $(A,«a) (2.10) 
Notice that since $(A, ■) is strictly contracting, any sequence w n G X satisfying 

w n = <J>(A, iu n _i) (2.11) 
converges strongly to as n goes to +oo. 

2.2 Shapley operators 

An important application, which is our main motivation, is obtained in the framework of 
zero-sum two player repeated games [25]. For example take the simple case of a stochastic 
game with a finite state space f2, compact move sets U and V for player 1 and 2 respec- 
tively, payoff g from U x V x fl to M, and transition probability p from U x V x fl to 
A(fl) (the set of probabilities on fl). Let S = Af(U) (resp. T = Ay(V)) the sets of 
probabilities on U (resp. V) with finite support; we still denote by g and p the multilinear 
extensions from U x V to S x T of the corresponding functions. 

The game is played as follow: an initial stage uj\ G fl is given, known by each player. 
At each stage m, knowing past history and current state uj m , player 1 (resp. player 2) 
chooses cr G S (resp. r G T). A move a m of player 1 (resp. b m of player 2) is drawn 
accordingly to cr (resp. r). The payoff g m at stage m is then g(a m ,b m ,uj m ) and uj m +i, the 
state at stage m + 1, is drawn accordingly to p(a m ,b m ,uj m ). 

There are several ways of evaluating a payoff for a given infinite history: 

— - J2m=i 9™ * s the payoff of the n— stage game 

— A X) m =i(l ~~ ^)*~ 1 fl , m i s the payoff of the A— discounted game. 

For a given initial state uj, we denote the values of those games by v n (cu) and V\(uj) 
respectively; v n and V\ are thus functions from fl into R. 

Let T = {/ : fl — > M}; the Shapley operator J from T to itself is then defined by 
/ — * </(/), where J(/) is the function from f2 to M satisfying 



max min < q(a, r, a;) + f(w')p(oj'\cr, r, uj) > (2.12) 
<reA(r/) rgA(y) I J 

min max < gfer, r, a;) + > f(w/)p(c</|cr, r , ^0 ^ (2.13) 
^'^1 ^ J 
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Then J is nonexpansive on T endowed with the uniform norm. The value v n of the 
n-stage game (resp. the value v\ of the A-discounted game) satisfies relation (2.9) (resp. 



This recursive structure holds in a wide class of zero-sum repeated games and the study 
of the asymptotic behavior of v n (resp. v\) as n tends to +00 (resp. as A tends to 0) is 
a major topic in game theory (see [25] for example). Convergence of both v n and v\ (as 
well as equality of the limits) has been obtained for different class of games, for example 
absorbing games [14], recursive games [8], games with incomplete information [2], finite 
stochastic games [4] [5], and Markov Chain Games with incomplete information [22]. 

Even in the simple case of a finite stochastic game where the space T on which J 
is defined is M n , the Shapley operator J is only nonexpansive for the uniform norm £°°. 
In the case of a general Shapley operator J, the Banach space (which may be infinite 
dimensional) on which J is nonexpansive is always a set of bounded real functions (defined 
on a set Q of states) endowed with the uniform norm. As shown in [11] and [15], this lack of 
geometrical smoothness implies that the families v n and V\ may not converge. They may 
also converge to two different limits [16]. However the goal of the so called "Operator 
Approach" (see [24] and [26]) is to infer, from specific properties in the framework of 
games, convergence of both v n and v \ as well as equality of their limits. 

A closely related application, in the framework of discrete event systems, is the problem 
of existence of the cycle-time of a topical mapping [9] [10]. 

2.3 Associated evolution equations 

In the current paper we investigate a slightly different direction : the aim is to show 
that the sequence v n and the family v x defined in equations (2.9) and (2.10) behave 
asymptotically as the solutions of certain continuous-time evolution equations. This is 
interesting for at least three reasons: first, this implies that proving the convergence of 
v n or v\ reduces to study the asymptotic of the solution of some evolution equation. 
Second, even if the definitions (2.9) of v n and (2.10) of v\ may seem dissimilar since one 
is recursive and the other is a fixed point equation, we will see that the corresponding 
equations in continuous time are of the same kind, hence it gives an insight on the equality 
\\mv n = \imv\, satisfied for a wide class of games. Third, we will prove in the process 
some results of interest in their own right. 

Notice that equation (2.1) can also be written as a difference equation 



(2.10)). 



(K+i - V n ) = J(V n ) - v n 



(2.14) 



which can be viewed as a discrete version of the evolution equation 



U\t) = J(U(t))-U(t). 



(2.15) 



Similarly, equations (2.9) and (2.11) can be considered as discrete versions of 




(2.16) 
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and 

u'(t) =$(\,u(t))-u(t) (2.17) 

respectively. Notice that while (2.17) is autonomous, (2.16) is not. 

The asymptotic relation between solutions of (2.15) and (2.1) will be discussed in 
section 3. In that section we will a also prove some results about Eulerian schemes related 
to (2.15), which have an interpretation in terms of games with uncertain duration [19,20] 
in the case of a Shapley Operator. 

In section 4, we will study the asymptotic behavior of solutions of the non-autonomous 
evolution equation 

u\t) = $(A(t), u(t)) - u(t) (2.18) 

for some time-dependent parametrizations X(t), which in particular will cover both cases 
of equations (2.16) and (2.17). We will first prove that when \(t) = t the solution of (2.18) 
has the same asymptotic behavior, as t goes to +oo, as the sequence v n as n goes to +oo. 
We will then examine the case where the parametrization \(t) converges slowly enough 
to 0, establishing that the solution of (2.18) has then the same asymptotic behavior as 
the family v\ as A goes to 0. Finally, using our results in continuous time, we will study 
other dynamics in discrete time generalizing (2.9) and (2.11). Similarly to section 3, in 
the case of a Shapley operator these dynamics have an interpretation in terms of games 
with uncertain duration. 



3 Dynamical system related to the operator J 

Let us denote A = I — J; the operator A is m-accretive, meaning that for any A > both 
properties are satisfied: 

(i) \\x - y + XA(x) - XA(y)\\ > \\x - y\\ for all (x,y) el 2 . 

(ii) / + \A is surjective. 

This implies that A is maximal monotone [12]. Recall that the analogous in continuous 
time of equation (2.1) defining V n is evolution equation (2.15), which can also be written 

as 

U'(t) = -A(U(t)) (3.1) 

with initial condition U(0) = Uq, the Cauchy-Lipschitz theorem ensuring the existence 
and uniqueness of such a solution. 

Example 3.1 Following example 2.1, suppose J(x) = x + c. Then one has A(x) = —c, 
so U(t) = U + ct. 

This simple example shows that, as in discrete time where the true sequence to consider 
is not V n but the normalized v n , we are not expecting convergence of U(t) but rather of 
the normalized quantity ^p-. This is a consequence of the fact that we do not assume 
non emptiness of v4 -1 (0). 
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Apart from equation (2.1), there are numerous other natural discretizations of equation 
(3.1). For every xq G X and any sequence {A„} in [Ojl] 1 the explicit Eulerian scheme is 
defined by 



n-1 



-X n Ax 



n-1 



(3.2) 



that is 



n[/-M] )(xo). (3-3) 



Notice that choosing xq = and A n = 1 for all n leads to the definition (2.1) ofV n . 

Other discrete trajectories are implicit proximal schemes (first introduced when A 
df in [18]) which satisfy: 

%n %n—l X n Ax n 



that is 

Xn= (nfJ + A^r 1 ) (Z0j 
\i=n 

In both cases we denote 



- E A * ( 3 - 4 ) 

i=i 

n 
i=l 

Usually proximal schemes share better asymptotic properties (take the simple example 
where A is a rotation in IR 2 and A„ ^ £ 2 : then the proximal scheme will converge to the 
fixed point of the rotation, while the Eulerian one will diverge). However Eulerian schemes 
have the remarkable feature that they can be computed explicitly, and they arise naturally 
in the game-theoretic framework: 



Example 3.2 When J is the Shapley operator of a stochastic game T, x n defined by (3.2) 
is the non-normalized value of the following n— stage game: states, actions, payoff and 
transition are as in V , but at stage 1 there is a probability 1 — X n that the game goes on 
to stage 2 without any payoff or transition. Similarly at stage 2, there is no payoff nor 
transition with probability 1 — A n _i ; and at stage n with probability 1 — Ai. In that case 
a n and r n have a nice interpretation: the expected number of stages really played is o n , 
and the variance is o~ n — T n . It is also worthwile to notice that such games are particular 
cases of stochastic games with uncertain duration [19,20]. 

For this reason we will study exclusively Eulerian schemes, in the case of an operator 
A — I — J. Results of this section will be of three kind: first we study the relative behavior 

1 Usually these schemes are defined for any sequence of positive steps, but here, since we need the 
operators / — \ n A to be non expansive, we have to assume that the A„ lie in [0, 1] 
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of continuous and discrete dynamics when time goes to infinity. Given a sequence A n ^ i 1 
one investigates the asymptotic relation between U(a n ) and the n-th term x n of the 
Eulerian scheme defined in (3.2). This is done first in the special case of V n (Corollary 
3.8) and then in general (Corollary 3.13). 

We also consider the case of a fixed time t. In that case one cuts the interval [0, t] in a 
finite number m of intervals of length Aj. These steps define an explicit scheme by (3.2), 
hence an approximate trajectory by linear interpolation. One expects such a trajectory to 
be asymptotically closer to the continuous trajectory defined by (3.1) as the discretization 
of the interval becomes finer. This is proved first in the case where A» = — for 1 < i < m 
(Proposition 3.10), and then generalized in Proposition 3.14. 

In the process we prove that two classical results, involving proximal schemes and 
holding for any maximal monotone operator, have an Eulerian counterpart when A is 
of the form I — J: we establish a new exponential formula in Proposition 3.10 and a 
Kobayashi-like inequality in Proposition 3.11. 

3.1 Asymptotic study of the trajectory defined by equation (3.1) 

The study of the asymptotic behavior of the solution of equation (2.15) in general Banach 
spaces has started in the early 70's, in particular the main result of this subsection, 
Corollary 3.8 relating v n and is already known (see [17] and [3]). Here we prove it 
in a different way, similar to the first chapter of [6], establishing during the proof some 
inequalities that will be helpful in the remaining of the paper. 
Let us begin by proving several useful lemmas: 

Lemma 3.3 Let f be a continuous function from [a, b] C E to E such that for every 
t e [a, b] 



for some continuous function g and some non-negative measurable function (3 such that 
j b a (3(s)ds < +oo. 
Then f satisfies 



for all t G [a, b}. 

Proof. Define a(t) = M + J t a g(s)ds. Since f(t) < a(t) + /* P(s)f{s)ds, Gronwall's 
inequality( [27] p. 15) implies that 





f(t) <a{t) + etiP {s) 




a{s)(3{s)e-^ f3{r)dr ds 



Integrating by part the last integral gives 
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fit) < a{t) + e^ p{s)ds ■ [a(0)-a(t)e-f^ (s)ds + / a' (s)e~ ^ p{r)dr ds 



In the remaining of the paper we will repeatedly use the following consequence of 
Lemma 3.3: 

Proposition 3.4 If y : [a, 6] C R — > X is an absolutely continuous function satisfying 
for every t G [a, b] 

\\y(t) + At)\\<(l-7(mv{t)\\ + Ht) 

where 7 is a continuous function from [a, b] to [—00, 1] and h is a continuous function 
from [a, b] to R, then y satisfies 

\\y{t)\\ < e -^ s)ds Uy{a)\\ + j\{s)e^ dr ds 
for all t G [a, b]. 

Proof. z(t) = y(t)e l satisfies ||z'(*)ll — (1 — 7(*))lk(^)ll + h(t)e t , hence for every t G [a, b] 



\z(t)\\ < \\z(a)\\ + 



z'(s)ds 



a 



ft 

< Ikfalll - I \\z'(s)\\ds 



< Ma)\\ + 

Applying Lemma 3.3 to thus gives 

\\z(t)\\ < e £ 1 -*' )da ( \\z(a)\\ + I h{s)e s e-^ 1 -^ r)dr ds 



Multiplying each side by e * implies the result. ■ 
We now use this technical result to compare two solutions of (2.15): 

Proposition 3.5 If both U and V satisfy (2.15), then \\U(t) — V[t)\\ is non-increasing. 

Proof. Define f = U — V which satisfies 

+ = \\J(u(t)) - J(V(t))\\ < \\u(t) - V(t)\\ = 

Apply the preceding proposition to 7 = and /. 
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Corollary 3.6 If U is a solution of (2.15), then ^ s non-increasing. 

Proof. Let h > and Uh{t) = U(t + h). The function C/j, satisfies equation (2.15), 
so applying the preceding proposition to U and Uh we get that t — > H^" 1 "^"^)!! is non- 
increasing on M + . Letting h go to gives the result ■ 



An interesting consequence of Corollary 3.6 is the following inequality, proved in Chap- 
ter 1 of [6]: 

Lemma 3.7 (ChernofP s estimate) Let U be the solution of (2.15) with U(0) = Uq. 
Then 

\\U{t) -J n (U )\\< \\U\0)Wt+ [n-e 

Sketch of proof. Proceed by induction on n; the proof for the case n = comes from 
the fact that ||£7'|| is non-increasing by Corollary 3.6. 

■ 

In particular if we take Uq = and t = n in Lemma 3.7, we finally get the following 
corollary relating continuous and discrete trajectories: 

Corollary 3.8 The solution U of (2.15) with U(0) = satisfies 

U{n) 
n 

In particular v n converges iff converges, and then the limits are the same. 

Proof. The only point that remains to be shown is that if the sequence converges 
as n tends to +oo, then so does as t tends to +oo. 
Using Corollary 3.6, we obtain 

\\U(t)-U([t})\\<(t-[t])\\U'(0)\\<\\U'(0)\\ 

which implies that — ^r|p goes to as t tends to +oo. 



< 



ll^(o)l| 



n 



3.2 An exponential formula 

When A is a m-accretive operator on a Banach space, a fundamental result (see [7] p. 
267) is that the solution U of (2.15) satisfies the following exponential formula for every 
t > 0, where the convergence is strong: 

lim (i + -a) (U ) = U(t) (3.6) 

In the special case where J is a nonexpansive operator and A = I — J, we now establish 
an Eulerian analogous of this classical "proximal exponential formula". 
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Definition 3.9 For x G X , I G N and t G R + , let us denote 



ur(x) 



m 



x 



(3.7) 



the m-th term of an Eulerian scheme with steps — 



Proposition 3.10 Let U G X and U the solution of (2.15) with U(0) = U . Then if 
m >t, 

\\U?(U )-U(t)\\<\\A(U )\\4=- (3.8) 



■in 



In particular, for any t > 0, the following strong convergence holds: 



t 



m— >+oo 



711 



lim I A (U ) = U(t) 



(3.9) 



Proof. For any A G [0, 1], J\ := A J + (1 — \)I = I — XA is nonexpansive. Denote by U\ 
the solution of 

U x {t) + U' x {t) = J x (U x {t)) (3.10) 
with U\(Q) = Uq. Applying Lemma 3.7 to U\ and the nonexpansive operator J\\ 



\W) - JZ(U )\\ < \\U' x (0)\\\/t + [n - t]< 



so in particular for n = t 



\U x (n) - J^(U )\\ < WU'MWV^- 



(3.11) 



Denote by U the the solution of (2.15) with £7(0) = Uq and notice that the function 
t —> U(Xt) satisfies (3.10) and has the same initial condition as U x . This implies that 
U x (t) = U(Xt) and putting this in (3.11), 

\\U(Xn)-JZ(U )\\<X\\U'(0)\\Vn-. 

For any t' < n, choosing A = - G [0, 1] thus gives 



U(t>) 

which is the desired result. 



if 



n 



A) (U ) 



< \\U'(0)\\ 



t' 



ii 



3.3 Comparaison of two Eulerian schemes 

To generalize Proposition 3.10 to explicit schemes with arbitrary steps, it is useful to 
estimate first the difference between two Euler schemes: let x and x in X, {A} n and 
{A n } two sequences in ]0,1]. Define x n , a n and r n (resp. x n , a n and f n ) as in (3.2), (3.4) 
and (3.5). The following proposition, which gives a majoration of the distance between 
two Eulerian trajectories, is an analogous of the classical Kobayashi inequality (Lemma 
2.1 in [13]) which gives a majoration of the distance between two proximal trajectories: 
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Proposition 3.11 For any z G X and (k,l) G N 2 , 



\\x k - xi\\ < \\x - z\\ + \\x - z\\ + \\A(z)\\>J(a k - Oi) 2 + T k + Ti 

Proof. We proceed by induction and begin by the case / = 0. 
We recall that J\ = I\A is non-expansive for A < 1, so we obtain 

\\xj-z\\ < \\xj- J Xj (z)\\ + \\J Xj (z) - z\\ 

= WJ^Xj-J- J Xj (z)\\ + XjWAWW 
< Hx^x-^ll +X J \\A{z)\\ 

and summing these inequalities for % G {1, • • • , k} we get \\x k — z\\ < \\x — z\\ + a k \\A(z)\\, 
which implies that 

\\x k -x \\ < \\x k - z\\ + \\x - z\\ < \\x - z\\ + \\x - z\\ +a k \\A(z)\\ 

and the proposition holds when 1 = 0. The case k = is proved in the same way. 

We will now assume the formula to be true for (k — 1,1), (k, I — 1) et (k — 1, 1 — 1) and 

deduce that it also holds for (k,l). 

Define numbers a k j = z^tat\ > Pk,i = A ' ( 3~ Afc \ et j k i = , Jf \ ? and note that 

they are non-negative with sum 1. Introduce also c kj i = \/ (cr k — a{) 2 + r k + 77. For any x 
and y in X, one check that the following equality holds: 

■h k ( x ) - J Xl {y) = ay(4W - v)+M x - J Xl {y)) + ik,i(J{x) - J(y)). 

In particular, letting x = x k -i, y = 27-1 and using the non-expansiveness of J, we get 

\\x k - xi\\ < a k ,i\\x k - xi-t\\ + f3 kt i\\xk-i - xi\\ + 7fe,/||a; fe _i - xi-i\\ 
so by induction, 

\\x k - 22 1| < ||x - z\\ + \\x - z\\ + \\A(z)\\(a K ic k) i-i + /3 k jC k - h i + jkjCk-ij-i) 
< \\x - z\\ + \\x - z\\ + \\A(z) || a/ a k j + /3 k j + J k ,iy/a\i 
= \\x - z\\ + \\x - z\\ + \\A(z)\\y/a\i 

where we have denoted d k>l = ttfc.j^j-i + Pk,iCk-i,i + 7fe,i c Li,z-i- 
In addition, 



and similarly, 



Ofc - ^-i) 2 + n + fj_i 

(cr fc - 6"; + A/) 2 + r fe + f{_i 

(o-fe - 07) 2 + A 2 + 2Xi(a k - 07) + r k + fj_i 
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Moreover 



= (cr/c - o x + \i - A fe ) 2 + r fc „i + f z _! 

= (crfe - en) 2 + Af + A^ + 2(A ; - A fe )(o- fc - 5-/) - 2A ; Afe + r fe _i + fj_i 
= 4, + 2(A,-A fc )( < 7 fc -CT,)-2A,Afc. 

So 

= + 2(o-fc - o-j)(a fc) iAj - /9fe,«Afc + 7fc,/(A; - A fc )) - 2A z Afc7w 

= + 2- — ^4 — — . (A fc A;(l - A/) - A fc A;(l - A fc ) + A fe Aj(A; - A fc )) - 2A;Afc7fc,« 
Afc + A/ — AfcA/ 

= 4,i ~ 2AjAfc7*,i 
< 4 

and we have established that 

- < \\xo - z\\ + ||x - z\\ + ||A(z)|| yVfc - &i) 2 + r fc + f/. 



3.4 Comparaison of an Eulerian scheme to a continuous trajec- 
tory 

We now combine the results of the two preceding subsections: Proposition 3.10 comparing 
the continuous trajectory with a particular Eulerian scheme, and Proposition 3.11 relating 
any two Eulerian schemes. 

Corollary 3.12 Let {x n } n eN be an Eulerian scheme as defined in (3.2). Then for any 
t > and k EN, 

\\x k -U{t)\\ < \\x -U \\ + \\A(U )\W(<T k -t)* + T k 
Proof. Apply Proposition 3.11 to x k and U^iUo) to get 



\\x k - U™(U )\\ < \\x -U \\ + \\A(U )\\)J (a k - tf + 

Let m go to +oo and use Proposition 3.10. ■ 
This corollary has some interesting consequences in two directions, as it generalizes 
both Corollary 3.8 and Proposition 3.10. First, it shows that any normalized discrete 
trajectory behave as the normalized continuous one as time goes to infinity: 
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Corollary 3.13 For any t > and any Eulerian scheme {xi} such that a k = t, 

\W-U(t)\\ < ||zo-^o|| + ||A(C7o)||v^ 



t ~ t 

Proof. Apply Corollary 3.12 and use the fact that r k < a k since all A» are in [0, 1]. ■ 

On the other hand, take now the case of a fixed time t. Let U be the solution of 
(3.1) with initial condition U(0) = Uo, and let {xi}o<i< n defined by (3.2) be an Eulerian 
scheme with same initial condition xq = Uo and o~ n = t. One constructs an approximation 
x of the continuous trajectory U on the interval [0, t] by x(a k ) = Xk for < k < n, 
and linear interpolation on intervals [a k , a k+ i]. The following proposition states that such 
approximation x will becomes asymptotically close to U as the discretization < Ai < 
Ai + A2 < • • • < cr n _i < a n = t of the interval [0, t] gets finer: 

Proposition 3.14 For any t' in the interval [0,t], 



\\x(t f ) - U(t>)\\ < \\A{U Q )\\{\ + (1 + V2)t) ■ ./max {A,}. 

y l<i<n 

Proof. Let t' G [0,t] and k such that a^-x <t'< a^,- Noticing that 

\W) - U{t')\\ < \\x{t') - x{a k )\\ + \\x{a k ) - U(a k )\\ + \\U(a k ) - U(t')\\ 

we will evaluate the three components of the right-hand side separately. 
Since x is affine on [<7fc,<7fc+i], applying Proposition 3.11 gives 

\\x(t') - x{a k ) || < ||z(«7 fc _i) - x(cr fe ) || 
ll-^fc X k — 1 (I 
< \\A(U )\\y/(a k - <J k ^f + r k + r fc _x 
= \\A(U )\\V2F k . (3.12) 

On another hand, Corollary 3.12 implies that 

\\x(cr k ) -U(a k )\\ = \\x k -U(a k )\\ 

< \\A{U Q )\\^F k . (3.13) 

Thirdly, using the mean value Theorem as well as Corollary 3.6, 

\\U(a k )-U(t')\\ < \a k -t'\ max \\U'(t")\\ 

t"e[t',a k ] 

< \a k -a k -i\-\\U'(0)\\ 

= X k \\A(U )\\. (3.14) 
Adding inequalities (3.12) to (3.14) we thus deduce that 

\\x(t') - U(t')\\ < \\A(U )\\(X k + (1 + y/^y/n). 



We use the facts that A^ < yAfe < y maxi<j< n {Aj}, and that r k < r n < t maxi<j< n {Aj} 
to conclude the proof. ■ 
This proposition has an interpretation in the particular framework of Example 3.2: 
consider a game with an expected duration of t. The previous result establishes that 
this game has a non normalized value close to U(t), providing that at each stage the 
probability of playing is small (that is to say, if there is a high variance in the number of 
stages really played). 
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4 Dynamical systems linked to the family $(A, ■) 



Let A : R — >]0, 1] be a continuous function. In this section we study the asymptotic 
behavior of the solution to evolution equation (2.18): 

u{t) + u'{t) = $(A(t), u(t)) with w(0) = u 

where $ is the operator defined by equation (2.7). 

Remark 4.1 Since the mapping (x, t) — > $(A(t), x) — x is globally 2-Lipschitz in its first 
variable, Cauchy-Lipschitz-Picard theorem ensures the existence and uniqueness of the 
solution of (2.18), and that it is defined on the whole set K + . 

When the recession function $(0, •) exists, any accumulation point v of v n or V\ will 
satisfy 

$(0,v)=v (4.1) 

but equation (4.1) may have many solutions (for example in the case of games with 
incomplete information [24] any convex/concave function satisfies (4.1)). The evolution 
equation (2.18) may thus be seen as a perturbation of (4.1), and we will study the effect 
of some perturbations on the asymptotic behavior of the solution of (2.18). See for 
example [1] for a similar approach in the framework of convex minimization. 
The main results of this section are the following: 

— When A is the constant A, the solution of (2.18) converges to V\. 

— When \(t) ~ t, the solution of (2.18) behave asymptotically as the family {v n } 

— When \(t) converges to slowly enough, the solution of (2.18) behave asymptoti- 
cally as the family {v\} 

The first two results are not surprising since in those cases evolution equation (2.18) is a 
continuous version of equation (2.11) or (2.9) respectively. The third result is of a different 
nature but is also natural: denote by u\ the solution of (2.18) when A is the constant A. 
We establish that if the parametrization A in (2.18) is of slow variation, the solution u 
evaluated at time t is close to u\(t)(t), hence to Vx(t) ( see figure below). 

In the process of proving those three results, we also answer natural questions about 
the behavior of the solution u of equation (2.18) as a function of the parameters, namely 
we will prove that: 

— If A ^ i l the asymptotic behaviour of u does not depend on the initial value u . 

— If two parametrizations A and A are asymptotically close, then it is also the case 
for the corresponding solutions u and u. 

First we prove a simple fact that will be repeatedly used in the remaining of the paper. 
Recall, by equation (2.10), that for any t > 0, V\(t) is the only solution of 

v Ht) = *(\(t),v m ). (4.2) 

The following Lemma relates the behavior of u'(t) to that of u(t) — v\(t)'- 
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Lemma 4.2 Let u be the solution of evolution equation (2.18) and be defined by 
(4.2). Then for any t > 0, \\u{t) - v m \\ < ™l 

Proof. 

\\u\t)\\ = Kf)-*(A(f),ti(f))|| 



\\u(t) 


-v Mt )\\ 


jju(t) 


-v M t)\\ 



\(t) \\u(t) - V X (t)\ 



4.1 Constant case 

We start by considering the simplest case where the function A is a constant A. Equation 
(2.18) is then a continuous analogous of equation (2.11), so one can expect that u{t) 
converges to t>A, and indeed this is the case. 
Start by a technical lemma: 

Lemma 4.3 If f satisfies f(t) + f'(t) = B(f(t)), where B is anl — X contracting operator, 
then 



\\f'(t)\\< ILf(0)|| -e 



-XI 



Proof. Let h > and fh(t) = ^ — - — ^— . Since B is (1 — A) contracting: 

\\A(t)+m\\ = l\\f(t+h) + f(t+h)-[f(t) + f(t)}\\ (4.3) 

= l -\\B{f{t + h))-B{f{t))\\ (4.4) 
< (l-A)||A(t)||. (4.5) 
Proposition 3.4 applied to fh thus implies that 

IIAWII<IIA(o)||-e- A * 

and letting h go to gives the result. 

■ 

An immediate consequence is: 
Corollary 4.4 If u is the solution of (2.18) with \{t) := X, then 

lim u(t) = v\ 

t— >+oo 



Proof. Lemmas 4.2 and 4.3 imply that 



e A * 



\\u{t) -v x \\ = \\u{t) -v Kt) \\ < < ||m'(0)|| 



\{t) ~ " v y " A 
and the right member goes to as t tends to +oo. 
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4.2 Some generalities on the non-autonomous case 

The case when the parametrization A is not constant is more difficult to handle: the same 
method as in the proof of corollary 4.4 leads to 

u(t + h)- u{t) + u\t + h)- u\t) = $(A(t + h), u{t + h)) - $(A(t), u{t)) 

but Proposition 3.4 does not apply. 

However, we can prove if the perturbation is strong enough: 

Proposition 4.5 If J^°° X(t)dt = +oo. the asymptotic behavior of u solution of (2.18) 
does not depend of the choice ofu(0). 

Proof. Let u and v be two solutions of (2.18), define the function g by g(x) = \\u(x) — 
v(x)\\. According to proposition 3.4, 

g(x) < g(0)-e-%W dt 
from which the proposition follows. ■ 

4.3 Case of \{t) ~ \ 

When A(£) = t, equation (2.18) is the continuous conterpart of equation (2.9), so we 
expect u{t) to have the same asymptotic behavior as v n . This will be proved with an 
additional hypothesis on $ in the next section. Here we show a slightly weaker result 
without any assumption. 

Proposition 4.6 There exists a function A : [0, +oo] — >](), 1] such that A(t) ~ t and for 

which the solution w of (2.18) satisfies 

||u>(n) — v n \\ — > 0. 

n— >+oo 



Proof. 

T7 cnln + inn nf (0 1.^1 onrl ill 1 — 

t+1 

(t + 2)v(t) + (t+ l)v\t) = J((t + l)v(t)). 



Let U be the solution of (2.15) and v (t) = j^j-, which thus satisfies 



Define ((t) = t + ln(l +t). By making the change of time s = ((t) and w(s) = v(t), we 
get 

w(s) + w'(s) = $ ( 2 + ^ 1(g) , w(s)j 
and w is thus solution of (2.18) with 

ua 1 i Ml (Mt)\ 
x{t) = 2Tc z W) = -t + — + {—)- 

Moreover, 

\\w(n) - v n \\ < \\v(n) - v n \\ + \\v (C _1 H) - v(n)\\ . 
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We already know by Corollary 3.8 that \\v(n) — v n \\ goes to as n tends to +00. On the 
other hand, by the mean value Theorem, 

\\v(C\n))-v(n)\\<(n-(-\n))- max \\v'(x)\\. (4.6) 

zG[C (n),n| 

(x + l)U'(x) - U(x) 

By definition of v, v'(x) = — ^. 2 hence Corollary 3.6 implies that 



W{x)\\ < y ^ ; "^7"7 " W " (4.7) 



(x + l)||^(x)|| + ||l/(s)|| 
+ 

(x + 1)11^(0)11 + ||*7(0)|| 

" (^+1? (48) 

< (4.9) 

x + 1 v ' 



for C = 2max(||C/(0)||,||C/'(0)||). 
Replacing in equation (4.6) gives 

||„(C-(»))-«(„)||<£7^gg 
which goes to since £(n) ~ n, and we have thus proved that 

Hn) -v n || — > 0. 

n— >+oo 



An interesting corollary of this Proposition, which gives a sufficient condition for 
convergence of both v n and v x to the same limit, is: 

Corollary 4.7 Let U be the solution of (2.15). If U'(t) converges to I when t goes to 
+oo, then v n and V\ converge to I as well as n goes to +00 and A goes to 0, respectively. 

Proof. Suppose that U'{t) converges to / when t goes to +00. Then v(t) = converges 
to I as well, and so does v n according to Corollary 3.8. 
On the other hand, 

tv'(t) = U'(t) - ™ _^ 1 _ 1 = 

so v'(t) = o (~). Define C, A and w as in the proof of the preceding proposition ; then 
w(t) = w(C _1 (*)) converges also to / and by definition 

«/(««)) = i±i^ ) = (I). 

Since £(£) ~ t and A(£) ~ | this implies that ^fiP = o(l). According to Lemma 4.2, this 
implies that — Ux(t) || — an d so V\ tends to Z as A goes to 0. 
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4.4 Case of a slow parametrization 

From now on the following assumption (H) will be made: there is a constant C such that 

||$(A,z) -$(//, z)|| < |A-//|(C+ ||x||) Viel V(A,/i) e]0,l] 2 - (W) 

Remark 4.8 is satisfied as soon as J is the Shapley operator (2.12) of a game with 
bounded payoff since in that case 

||$(A, x) - < |A - m | (ii^iu + i^iu) 

Remark 4.9 Hypothesis (TC) implies that for every A and \x 

IK - U|t|| < C_ 



\\ — H\ A 

/or some constant C : in some sense is thus a statement about the speed of variation 
of the family {v\\. 

The principal result of this subsection is Corollary 4.12 which states that under this 
hypothesis, if the parametrization A converges slowly enough to 0, then the corresponding 
solution of (2.18) has the same asymptotic behavior as the family {v\}. We start by a 
technical result: 

Proposition 4.10 Let A be a C 1 function from [0, +oo[ to ]0, 1] and let L : M + — > R be 

defined by L{t) = e L x(s) s \ s _ j , / len ^ e corresponding solution u of (2.18) satisfies: 

ii m „ £(*) r . . _ y* ia'Cs 



u'(0)|| + (C + C) / L F ffds 

M S J 



where C is the constant in condition (7i) and C = sup ||ua||. 

Ae]o,i] 

Proof. For any h > 0, define = — ^ — and A /l (t) = — ^ — , 

Since u is C 1 , 



1 



t+/i 



Ufc (t) = u'(t) + - / u'{s) - u\t)ds 
"> Jt 

which implies, by uniform continuity of u on any compact set, that the restriction of Uh 
to any closed intervall converges uniformly to u' as h goes to 0. Similarly, the restriction 
of Xh to any closed intervall converges uniformly to A' as h goes to 0. 
Since u satisfies equation (2.18), for any h and t, 

\\u h (t)+u' h (t)\\ = l\mX(t + h),u(t + h))-HX(t),u(t))\\ (4-10) 

< ~ \m\(t + h),u(t + h)) -®(\(t + h),u(t))\\ (4.11) 

+±\\$(\(t + h) 1 u(t))-$(\(t)(t) 1 u(t))\\ 

< (1 - X(t + h))\\u h (t)\\ + \X h (t)\(C + |K*)||). (4.12) 



19 



by hypothesis (TC). According to Lemma 4.2, this implies that 

WMt) + w h (t)\\ < (i - \(t + h))\\u h (t)\\ + |A,(t)| (c + a + 



(4.13) 



where C is a majorant of the family \v\\. 

Fix t > 0, and let e > 0. Since A(t) is bounded from below on [0,to] an d using the 
uniform convergence of Uh to v! on [0, to], one obtains that for h small enough, and for 
every t < t , 



\u h (t)+u' h (t)\\ < [l-\(t + h) + 



\>*(t)\ 
A(t) 



\u h (t)\\ + (C + C' + e)\X h (t)\. (4.14) 



Then applying Proposition 3.4 to implies that for any h small enough and t < t , 



u h (t)\ < e J 



( \\ Uh (0)\\ +(C + C' + e) f \X h (s)\e^ r+h ^] dr 



ds 



Using the uniform convergence of and A(- + h) on [0, to], letting h go to implies that 
for any t < to, 



u'(t)\\<L(t)[\\u'(0)\\ + (C + C' + e) 



* lA'QQl 
/o L(s) 

Since this is true for every to an d £, using Lemma 4.2 again gives 



-ds 



\u(t) - V X (t)\\ < 



m 

A(t) 



K(o)|| + (c + C) 



1 1 \ / 



o Us) 



■ds 



Remark 4.11 If in Proposition 4.10 we suppose in addition that A is nonincreasing, we 
get the simpler inequality 



\u(t)- v X (t) || < 



- So A(s)ds 

a^T 



|m'(0)|| - (C + C) / \{s)\'{s)efo x{r)dr ds 
Jo 



As a corollary to Proposition 4.10 we can now prove: 

Corollary 4.12 Let A be a C 1 function from [0, +oo[ to ]0, 1], such that converges 
to as t goes to +oo, and let u be the corresponding solution of equation (2.18). Then 
\\u(t) — V\m\\ goes to as t goes to +oo. 



Proof. First notice that ^x^yj — (1) ? s ° x^y = o(t) which implies that A(t) ^ i 1 . 
Next we prove that 

L(t) 



A(t) 



o(l). 
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ft\ \X'(s)\ X'(s) 
gJO [ A(s) A(s) 

Since the left-hand side is equal to 



\(s)\ds 



the result is deduced from the 



fact that = o(A(s)) and that A(t) i f. 



Finally we prove that 




Lit) 



\{t) 



) 



Since the right-hand side diverges to +00, it is enough to prove that the derivative satisfies 



Remark 4.13 Note the similarity of this proposition with some approximation results for 
dynamical systems in the framework of Hilbert spaces, for example the slow parametriza- 
tion in [lj: 

— first there is a parallel between the strong monotonicity condition in [1] p. 523 and 
our assumption that the $(A, •) are contracting. 

— Second between a condition about the derivative of the trajectory in the same paper 
p. 528 and our hypothesis (see remark 4.9). 

— Third the slow- convergence condition is the same (see condition (ii) in [1] p. 528). 

— Lastly, results of both papers are of the same nature: convergence of a certain fam- 
ily ({v\} in this paper) implies that the solution of any slowly-perturbed evolution 
equation tends to this limit as time goes to infinity. 

A difference however is the fact that in this paper we also have a reciprocal: if for any 
slow parametrization A the solution u(t) of (2.18) converges as t goes to infinity, then the 
family v\ converges to the same limit as A goes to 0. 

Remark 4.14 In the proof of Proposition 4- 10 only the three following hypotheses on the 
family $ were used: 

(i) $(•, x) satisfies condition Ti for all x. 

(ii) $(A, ■) is 1 — A contracting for every A e]0, 1]. 

(iii) The fixed points V\ are uniformly bounded. 

The two last ones are satisfied as soon as $(A, x) = A J (^j^x) for a nonexpansive operator 
J, but this is not a necessary condition for Proposition 4-10 to holds. 

Remark 4.15 In fact, the more general result holds: suppose that the family $ satisfies 
the three hypotheses: 




= o 




which is true since A'(t) = o(A 2 



(*))■ 
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(i) There exists a constant C and a continuous function M from ]0, 1] to M + such that 
for any (x,X,fi) inlxjO,!] 2 , 



|$(A,x)-$(/x,x)|| < 



M(7)d7 



A 



(c+Ni). 



(ii) There exists a continuous function (3 :]0, 1] —►](), 1] ,swe/i i/iai $(A, •) «s 1 — /3(A) 
contracting. 

(iii) The fixed points V\ o/$(A, •) are uniformly bounded by C . 

Let X be aC l function from [0, +oo[ to ]0, 1]. Then the corresponding solution u of (2.18) 
satisfies 

IK*)-^t)|| < 

f l r s \nt\r\\ |A'(i)|iVf (A(i)) 1 

u'(0)|| + (C + C) / |A'(s)|M(A(s))e Jo ^^\ dl ds 



e 



(3(X(t)) 

This implies that \\u(t) — V\(a\\ tends to as soon as (3 is C 1 and the parametrization A 
satisfies both properties : 

(-) ^SSF = 

(v) ^F = (i) 

Notice again the similarity with [1]. 

Another interesting consequence of hypothesis (H) is Corollary 4.17 which states that 
if two parametrizations are close to one other, then this is also the case for the trajectories. 
We first prove a technical result using the same approach as in the proof of Proposition 
4.10: 

Proposition 4.16 Let u and v be the two solutions of (2.18) for some functions X and 
fi respectively. Then for any t > 0, 

\\u(t) - v(t)\\ < e~ti^ s)ds {\\uq- vq\\ + (C + \\u(s)\\)\X(s) - fj,(s)\ ■ ef° »®di ds 
Proof. Let f — u — v, then 

\\f(t)+f(t)\\ = \mx(t),u(t))-®(rtt)xt))\\ 

< ||$(A(£), u(t)) - $(>(*), u(t))\\ + \M»(t), u(t)) - $(/*(*), v(t))\\ 

< \X(t)-»(t)\-(C+\\u(t\\) + (l-„(t))-\\f(t)\\ 

because of hypothesis (H) and contraction of $(A, •). Applying Proposition 3.4 gives the 
result. ■ 
In particular one has: 
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Corollary 4.17 Let u and v the two solutions of (2.18) for some functions X and \x 
respectively. Assume that u is bounded and /x ^ i 1 , then \\u(t) — v(t)\\ — > in the two 
following cases: 

a) n{t) ~ \{t) as t goes to +oo 

b) |A-a*| ei 1 . 

Proof. Let L be a bound for u. By the preceding proposition we know that 

\\u(t) - v(t)\\ < e -/o*»)* (\\ Uo - v \\ + (C + L) j* \\{s) - n(s)\ ■ eXrtWds 
so it suffices to show that 

Jo **(■ 



\\{s) - n{ 8 )\ ■ e^> {l)di ds = o ( e J 



a) Assume that fi(t) ~ X(i), that is = This implies that 

\X(t) - fi(t)\ ■ efirt' )da = o ** w<fa 

which gives the result by integrating. 

b) Assume that |A — fj,\ E i 1 , let e > and T such that J^°° \X(s) — fi(s)\ds < e. Then 
for t > T, 



/ \X(s) -ef>® di ds 
Jo 

[ \X{s) - /z(s)| • e^^ )di ds + f |A(s) - n(s)\ ■ e^^ )di ds 

Jo Jt 



t f r 

< e / / | A ( S ) _ ^^ds + e ti» {s)ds / |A(s) - /z(s)|ds 

Jo Jt 

< e ti^ s)ds I |A(s) - ^(s)|ds + seti^ s)ds 

Jo 

for all t large enough since ef° v( s ) ds diverges to +oo as t goes to +oo. 

■ 

Some interesting corollaries follows immediately: first because of Corollary 4.4, we get 

the 

Corollary 4.18 If X(t) -> A > 0, then u{t) -> w A 
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Then, combining the results of section 4.3 and Corollaries 4.12 and 4.17 we deduce 
the following Corollary bringing to light the tight difference between dynamics related to 
\imv n and \imv\: 

Corollary 4.19 For a G [0, 1[, let u a be the solution of 

u(t) + u\t) = $ ((1 + t) a ~\u(t)) withu(0) = u (4.15) 

Then u°(t) converges to some I G X when t goes to +oo iff v n converges to I as n goes to 
+oo ; and for a g]0, 1[ u a {t) converges to some I G X as t goes to +oo iff v\ converges 
to I as A goes to 0. 



4.5 Back to discrete time 

We proved in the last section that under hypothesis (TC), the solution of (2.18) has the 
same asymptotic behavior as the family {v\} as soon as A converges slowly enough to 0. 
One may wonder if it is true as well in discrete time. For any sequence (A n ) nS N in ]0, 1], 
define the discrete counterpart of equation (2.18) : 

w n = $(A n , w n -i) with w(0) = w (4.16) 

Then one obtains the discrete version of Corollary 4.12 : 

Proposition 4.20 Let \ n be a sequence m]0, 1]. Assume that both X n and j — ^— j- tend 
to as n goes to +oo. Then the solution {w n ) n ^ of (4-16) satisfies 



IK n - w n \\ 

as n goes to +oo. 

Proof. The sequence 7n = x~ tends to +oo and satisfies 7„ — 7„_i — > as n goes to 
+oo. This implies the existence of an interpolation function 7 : R — ► E which is C 2 and 
such that for all n in N, 7(71) = 7™ , lim +00 7(t) = +00 and lim +00 7'(t) = 0. The function 
A := - thus satisfies A(n) = A„ and all the hypotheses of Corollary 4.12. Let us denote by 
u the corresponding solution of equation (2.18). By Corollary 4.12 it is enough to show 
that \\w n — u{n)\\ — > as n goes to +00. 

Define a n := \\w n — u(n)\\ and let s > 0. Then 

a n = \\$(\ n ,w n -i) - $(\n,u(n)) + u'(n)\\ 

< (l-A n )|K_!-«(n) || + \\u'(n) || 

< (1 - X n )\\w n -i - u{n- 1)|| + \\u{n) -u(n- 1)|| + \\u'(n)\\ 

< (1 — A n )a n _i + 2 sup ||«'(£)|| 

te[n— l,n] 

< (1 - A„)a n _i + 2 sup 

te[n— l,n] 

< (1 - A n )a n _! + 2e sup A(t) (4.17) 

t£[n— l,n] 



«'(*) 



A(t) 



sup \(t) 

tGfn-l.nl 
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for every n large enough because of Corollary 4.12. 

Denote s n = max \(t) = o(l), and choose t n G [n — l,n] such that s n = X(t n ). Let 

te[n— l,n] 

T > such that |A'(i)| < A 2 (t) for every t > T, then by the mean value Theorem, for 
any n>T+l, 

\s n -X n \ = \X(t n ) - \(n)\ 

< \t n — n\ ■ sup |A'(t)| 

te[n-l,n] 

< sup \ 2 {t) 

te[n—l,n] 

= sl 
= o(s n ) 

so that s n ~ A n as n goes to +00. Together with (4.17) this implies that there exists N 
such that for all n> N ', 

a„. < (1 - A n )a n _i + 3eA„ 
and so by induction one prove that for all fceN, 

fc 

ajv+fe - 3e < (a N - 3s) J|(l - Ajv +i ) 

i=l 

Now j- — y^~7 — > implies that ~ = o(A n ), so the product goes to and we deduce that 
a-N+k < 4e for k large enough. ■ 

Corollary 4.21 V\ converges as A goes to if and only if there exists a sequence X n 
satisfying the hypothesis of Proposition such that the corresponding sequence w n 
defined by (4- 16) converges. 

Proof. Let A n such that w n converges. Because of Proposition 4.20, v\ n converges. 

Moreover, for all A and hypothesis (H) implies that, denoting C = sup 

Ae]o,i] 

\\v\-vj = \\$(\,v x ) ~ H^,v„)\\ 

< \\Q(fJi,vx) - QfavJW + ||$(A,i; A ) - $(fji,vx)\\ 

< (l-/i)||^-^|| + |A-/i|(C + C") 



\V\ - full < 



An 



(C + C). (4.18) 



and thus that 

A 
/' 

Since A n — > and ^ — — * ^ ~~ : T ±i l a l so converges to 0. Together with inequality 
(4.18) and the fact that v\ n converges it implies the convergence of v\ as A goes to 0. 

Conversely, if V\ converges, then Proposition 4.20 implies that the sequence w n defined 
by equation (4.16) converges as soon as A n and — x^T ^ enc ^ *° ^- ■ 
As in the section 3 (Example 3.2), there is an interpretation in terms of games with 
uncertain duration: 
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Example 4.22 Consider the case of a game with Shapley operator J. Let {A n } be a 
sequence in ]0, 1] and w n defined by equation (4-16). Then w n is the value of the following 
game with uncertain duration: with probability X n the game stops after stage 1, and the 
payoff is the payoff during stage 1. With probability 1 — A n there is no payoff during stage 
1 but a transition, and game goes to stage 2. Then, conditionally to the game going to 
stage 2, with probability A n _i the game stops after stage 2, and the payoff is the payoff 
during stage 2 ; and with probability 1 — \ n -i there is no payoff during stage 2 but a 
transition, and game goes to stage 3. If the game goes to stage n, with probability \\ the 
payoff is the payoff during stage n and with probability 1 — Ai the payoff is 0. 
Proposition 4-20 then states that «/{A„} is of slow variation, the value of this game with 
uncertain duration is close to the value of the A n - discounted game. 

As a final remark to this section, notice the way in which we proved Proposition 4.20, 
with a back and forth process to continuous dynamics ; it should be interesting to search 
another proof using only discrete time methods. 

5 Concluding remarks 

— In this paper we proved that the asymptotic behavior of v n and v\ can be derived 
from the asymptotic behavior of solutions of some evolutions equations, namely 
(2.15) and (2.17). It should thus be interesting to determine which additional con- 
ditions on the nonexpansive operator J may imply convergence of the solutions of 
these equations, and so convergence of v n and v\. 

— Notice that Corollary 4.19 hints that v\ and v n should have the same asymptotic 
behavior for a wide class of nonexpansive operators, since the study of limu n seems 
to be a limit case of the study of limt>A- Of interest is also Corollary 4.7 which gives 
a sufficient condition for existence of both lim v n and lim v\ as well as their equality. 

— In Examples 3.2 and 4.22 we saw that some results that arose naturally during this 
paper have a nice interpretation in the framework of games with uncertain duration. 
In particular we showed that for specific types of uncertain duration, the value of 
those games behave asymptotically either as v n or v \ as the expected time played 
tends to infinity. Following [19,20] it thus should be interesting to study uncertain 
duration more generally, hoping that some conditions on the Shapley Operator will 
provide convergence of values for more than just finitely repeated and discounted 
games. 
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